A Paradox of Semiparametric Estimators with Infinite Dimensional Nuisance Parameters

Pierce (1982) found a paradoxical phenomenon. Let θ = (β, γ) be parameters which we would like to estimate. In many cases, we are only interested in some parameters and the rest is nuisance parameters. Let β be parameters we were interested in and γ be nuisance parameters. Usually, an estimator of β has smaller variance when the nuisance parameter γ is known. Pierce (1982) found that under some conditions the variance of estimator of β with unknown γ is smaller than the one with known γ.